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ABSTRACT 


The Department of Defense (DoD) is making significant strides to develop and deploy 





unmanned vehicles in a variety of environments. Specifically, the Secretary of the Navy 
is sponsoring a new program, Consortium for Robotics and Unmanned Systems 
Education and Research (“CRUSER”), at the Naval Postgraduate School to enhance the 
ability to address unmanned vehicle research in a systematic manner. The area of 
research in this thesis strives to position the technological advancements within an ethical 
framework that will guide the development and use of these technologies. Autonomous 
platforms may bring significant advantages and enhance our abilities for mission 
accomplishment. This project concludes that they are best deployed in conventional 
conflicts, and may have more limited and problematic uses during irregular warfare and 
COIN operations. Laws pertaining to the deployment of autonomous and unmanned 
platforms are unclear and need to be strengthened on an international scale. Furthermore, 
the questions regarding what are permissible uses of autonomous platforms should also 
include future operators and personnel involved in the acquisition and engineering of 
these platforms, and should not be left solely in the hands of lawyers and diplomats. The 
combination of autonomy and lethality is found to work best when limited to the 
targeting of an enemy’s weapons systems and aircraft in highly scripted environments 


rather than enemy combatants and personnel themselves. 
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I. INTRODUCTION 


Imagine...if we had been able to wrestle with the great changes that 
atomic bombs brought to politics while they were being invented, rather 
than waiting to puzzle through their implications years later. 


Peter W. Singer, Wired For War! 


The prospect of using autonomous unmanned combat aerial vehicles (UCAVs) to 
support the nation’s military objectives would be a major paradigm shift in U.S. military 
operations. The reliance on artificial intelligence is quickly affording new abilities to 
wage war, and may not only result in deploying autonomous unmanned aerial vehicles, 
but also lethal autonomous unmanned land and sea vehicles as well. Autonomous 
platforms have the potential to “change the nature of warfare” and re-shape the force 
structure of our nation’s Armed Forces.? With the introduction of any new weapon 
system, however, comes the corresponding decision of how best it should be used. By 
deciding to develop and acquire autonomous platforms, we are simultaneously accepting 
a great moral and legal responsibility and are implicitly accepting its difficult legal and 
ethical questions. We need to ponder the question of how best to embrace the coming 
paradigm shift. 

The “brave new world’? of ethical and policy considerations that autonomous 
unmanned combat aerial vehicles (UCAVs) present today is a world in which the 
weapons may decide for themselves whom to target and when, possibly without human 
oversight and approval of their real-time strike decisions. Philosophers and ethicists argue 
that the ability of the autonomous UCAV to decide, in principle, whether or not to kill a 


human being, “elevates it ontologically and maybe even morally from the mere object to 


! Peter Singer, Wired for War, (New York: Penguin, 2009), 11. 


2 Armin Krishnan, Killer Robots: Legality and Ethicality of Autonomous Weapons (Burlington: 
Ashgate Publishing Company, 2009), 31. 


3 This phrase taken is from the title of Aldous Huxley’s novel, Brave New World, published in 1932. 
Huxley’s Brave New World, described a future with novel technological advancements in medicine while 
this research explores a new kind of warfare where machines make lethal decisions without human 
intervention. 


a subject capable of morally meaningful action.”* Armin Krishnan, for example, argues, 
“the concept of the lethal autonomous military robot is in some sense just the latest 
expression of a broader and quite disturbing trend in warfare, which is the general decline 
of human decision making.” 

The strength of the military has always depended on the capabilities and 
professionalism of the men and women in uniform. The military’s evolving reliance on 
artificial intelligence, however, may lead to a shift in our prevailing force structure to 
include autonomous platforms and other unmanned platforms working alongside 
traditional human forces. This could well change the emphasis from acquiring 
technologies that enhance our war-fighters to acquiring technologies that replace them. 
In any event, the military’s manpower requirements will almost certainly look 
significantly different than they do today and military personnel may assume new roles 
never previously conceived, even as their traditional roles vanish. 

One of the main drivers toward automation in the Armed Forces is to reduce 
personnel-related costs. Krishnan’s argument that “the ultimate goal behind the 
introduction of robotic systems is to reduce manpower requirements for military 
operations,” may not be completely valid, but if we are not careful in our strategic 
planning, it could become the sole driver.® 

The Secretary of the Navy is sponsoring a new program, “CCRUSER” (Consortium 
for Robotics and Unmanned Systems Education and Research) at the Naval Postgraduate 
School to enhance the ability to address unmanned vehicle (UV) research in a systematic 
manner. The CRUSER program encompasses a multitude of studies from various 
departments at the Naval Postgraduate School. The area of research in this thesis strives 


to position the technological advancements within an ethical framework that will guide 


4 Krishnan, Killer Robots, 33. 
5 Krishnan, Killer Robots, 33. 
6 Krishnan, Killer Robots, 35. 


the development and use of these technologies.’ Consistent with the Chief of Naval 
Operations ““duty to man, operate, and equip the fleet,” this research will examine how an 


evolving force structure will affect our future. 
A. RESEARCH QUESTIONS 


This thesis addresses the following research questions: 


1. To what extent do contemporary understandings of military ethics, the military 
profession, and the Laws of Armed Conflict (LOAC) address machine autonomy in 


conjunction with the use of lethal force in military conflict? 


2. What permissible uses of autonomous UCAVs might be envisioned within the 


present framework of warfighting? 


3. Is it necessary to retain a human presence, or executive oversight, in the UCAV 


“kill chain?” 
B. SIGNIFICANCE OF RESEARCH 


The U.S. Armed Forces have flown unmanned aerial vehicles (UAVs), and 
remotely piloted aircraft (RPAs), for reconnaissance missions for decades. They are 
rapidly gaining global attention for their offensive use in Iraq, Afghanistan, and Pakistan. 
The use of RPAs provides the Armed Forces a significant military advantage and 
provides a much greater degree of safety for our service members by removing them 


from the physical environment of the battlefield. 


The DoD is considering a number of concept UCAVs that may rely on increased 
degrees of autonomy. The current operational UCAVs include the Air Force’s Predator 
and MQ-9 Reaper, but many other concept UCAVs are still in development, both in the 
United States and abroad. The Navy is currently considering two “N-UCAS” (Unmanned 
Combat Aerial Systems) platforms. These are the UCAS-D and the X-47B. 
Approximately the same size as a modern jet fighter, the N-UCAS, could provide 


7 Captain Jeff Kline, USN (ret), Director of CRUSER at Naval Postgraduate School, assisted in 
framing the proposal for this thesis. 
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significant advantages to airpower and long-range strike capabilities, such as the “ability 
to engage and defeat a time-sensitive target in a matter of minutes owing to its 
persistence, sensor suite, multiple target capability, and kinetic or nonkinetic weapons 
systems.”8 The N-UCAS will have great force projection abilities, including “a wide 
array of air-to-air and air-to-ground munitions.”? The introduction of autonomy in the N- 
UCAS would make it possible to carry even heavier payloads and increase its power 
projection, although this may take years. 

Although work is already underway toward integrating autonomy in UCAVs, it 
may still be years before a U.S. military aircraft flies its own combat mission 
autonomously, relying only on artificial intelligence. This thesis will address some of the 
ethical and policy questions that will guide the use of such autonomous lethal platforms. 
It also proposes to stimulate and encourage others who are involved and will become 
involved in the acquisition, development and deployment of them to raise and consider 
such questions even further as they proceed in their vital work for the nation’s military. 

Likewise, this thesis will address strategic assumptions that need to be evaluated 
before unmanned systems assume many roles traditionally held by human personnel. The 
continual advent of new technology will make it possible to replace human personnel 
with autonomous systems in a wide variety of military roles besides aerial platforms. In 
turn, that research and discussion should influence the question of whether the military 
should in fact implement autonomous platforms into its force structure and how it should 
implement them. Even though the military possesses the technological capacity to 
develop and deploy new autonomous platforms, it does not necessarily follow that it is 
imperative that we do so. In other words, “just because we can, doesn’t mean we should.” 

Alterations in the force structure mix will change the strategic equation. We will 
inevitably think differently about strategy, if we succeed in combining autonomy and 
lethal force. The implementation of autonomous UCAVs will have far-reaching affects 
for the entire Armed Forces. Today’s environment provides an opportunity for the Armed 


Forces to engage in serious debate and research on the subject of autonomy and lethal 





8 Robert P. Haffa, Jr. and Michael W. Isherwood, “Long-Range Conventional Strike: A Joint Family 
of Systems,” Joint Forces Quarterly, 1st Quarter (2011): 104-105. 


9 Haffa and Isherwood, “Long-Range Conventional Strike,” 105. 
4 


force “in time to address them while still in the ‘design and development phase,’ when 
planning and governing might make a substantial difference.”!9 Research now on such 
questions could also save the U.S. Armed Forces resources that might otherwise be 


squandered on platforms with marginal utility. 
C; ORGANIZATION AND METHODOLOGY 


This research will examine the ethical framework for the use of lethal autonomous 
unmanned combat aerial vehicles. It consists of a qualitative review of classical moral 
arguments concerning the proper conduct of war, military professionalism, and the Laws 
of Armed Conflict (LOAC) as they relate to autonomy and lethal force in UCAVs. 
Chapter II includes a brief overview of the history of remotely piloted aircraft and their 
role in military history. It also highlights ethical dilemmas related to remotely piloted 
aircraft. Chapter IIJ examines the positive and negative ethical challenges associated with 
the introduction of autonomy into lethal UCAVs, including associated legal questions and 
considerations on the military professional ethic drawn from the Just War Tradition. 
Chapter IV discusses the likely impact of the introduction of autonomous lethal force. 
Chapters V presents recommendations on where it may be morally permissible to 
introduce autonomy into a UCAV platform and proposes questions the military needs to 


ask as it moves forward with this technology. 


10 George R. Lucas, Jr., “Nerds Gone Wild: Can Moore’s Law Remain Valid Indefinitely,” 
International Journal of Applied Philosophy 24 (2010): 79. 
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I. REMOTELY PILOTED AIRCRAFT 


A. BACKGROUND 


The quest to find an appropriate military use of unmanned aircraft began nearly a 
century ago, a little more than a decade after the Wright brothers flew the first plane at 
Kittyhawk. The idea of an “aerial torpedo” was proposed in Great Britain and a prototype 
was demonstrated to Army generals in 1917. !! Following the British prototype, the U.S. 
Navy and Army experimented with two of their own prototypes, but World War I ended 
before either was tested in battle.!2 Development continued in both nations and in 1937, 
the Navy’s Curtiss N2C-2 was introduced, a remotely piloted aircraft that could be 
controlled by another pilot at a distance up to twenty miles. In World War II, the 
Germans developed a V-1 “flying bomb” that was equipped with a jet engine and could 
travel over one-hundred fifty miles. The Germans launched eight thousand V-1s in the 
war, mainly against the British.!3 From an economical perspective, the V-1 was a success 
as the Germans “caused the Allies about three times the damage it cost the Germans to 
produce it,” however, it was plagued by “massive inaccuracy.”!4 The Japanese had 
actually experimented with similar prototypes to the V-1 before deploying suicide pilots. 
At the end of WWII, the U.S. Air Force launched modified B-17 and B-24 bombers 
“loaded with 9 tons of bombs and guided by remote control towards German targets, but 
all of them seemed to have either crashed or were shot down before they reached their 
targets.”!5 Undeterred, an Air Force General predicted in 1945, “the next war may be 


fought by airplanes with no men in them at all.”!6 


11 Krishnan, Killer Robots, 15. 
12 Krishnan, Killer Robots, 16. 
13 Krishnan, Killer Robots, 17. 
14 Krishnan, Killer Robots, 17. 
13 Krishnan, Killer Robots, 19. 


16S. M. Shaker and A. R. Wise, War Without Men: Robots on the Future Battlefield (Washington, 
DC: Pergamon-Brassey’s, 1988), 87 as quoted in Krishnan, Killer Robots, 19. 
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One of the first U.S. reconnaissance UAVs, the Red Wagon, was spurred by the 
Soviet’s downing of Francis Gary Power’s U-2 in 1960, however, the contract for the 
Red Wagon was shortly canceled. !? The Director of Defense Research believed the 
funds would be better spent on a manned reconnaissance aircraft. This was indicative of 
early attempts toward funding and building UAVs, which struggled to compete with 
research for manned aircraft and satellites.18 The National Research Office (NRO) did 
fund modification programs that converted aircraft to unmanned platforms. Four Q-2C 
special purpose aircraft were converted to stealth enhanced Model 147A Fire Fly drones 
that flew successful test missions in the U.S. from air launches under the wings of C- 
130s. While the Fire Fly struggled to find a permanent military home, another Cold War 


event led to a more realized need for unmanned platforms in military operations.!9 


In the lead-up to the Cuban Missile Crisis, a Soviet SA-2 shot down a U.S. U-2 
over Cuba. Following the shoot down, the Air Force deployed the Fire Fly force to 
Tyndall AFB in Florida to fly over Cuba. However, the mission was aborted because the 
Air Force was concerned that the Soviet Union would discover the U.S. capability for 
unmanned reconnaissance operations, but the proposed operation over Cuba led to a new 
contract for seven model 147Bs, a high altitude reconnaissance vehicle.29 The CIA 
wanted to use the new drones to gather information on the Soviet’s SA-2 missile system 
because it had the capability to electronically pass information over real time radio waves 


without the risk of jeopardizing human lives.?! 


The Fire Fly was renamed Lightning Bug in 1963 and performed its first 
operational deployment in 1964 after the Tonkin Gulf Incident. While they struggled 
initially, they eventually succeeded in monitoring Chinese operations. The Lightning 
Bugs were also used in Vietnam, which “became a testing ground for electronic warfare 


and automated command and sensor networks, or for what was later called the 


17 Thomas P. Ehrhard, Air Force UAVs: The Secret History (Arlington: Mitchell Institute Press, 
2010), 6. This section draws heavily from this report. 


18 Ehrhard, Air Force UAVs, 6. 
19 Ehrhard, Air Force UAVs, 7. 
20 Erhard, Air Force UAVs, 8. 
21 Ehrhard, Air Force UAVs, 8. 


‘automated battlefield.’“22 The 147T, Combat Dawn, was developed and deployed to 
Vietnam in the late 1960s as well. Developed for a photoreconnaissance and signal 
intelligence role, the 147Ts performed with high reliability rates until 1975. Politics, 
budgetary concerns, and particularly competition with satellites were responsible for their 
curtailment, but the platforms introduced in Vietnam paved the way for future 
“autonomous features.”23 Krishnan argues that: 

The smart weapons and sensors of the Vietnam era were generally not 

[autonomous weapons] in the sense that they could be automatically 

launched at targets, but they were already quite autonomous with respect 


to finding and attacking targets once they were launched by a human 
operator.24 


Motivated by China’s successful testing of nuclear bombs, another UAV, the D- 
21, was developed to fly roundtrip to China from Taiwan and drop its collected 
information via parachute before self-destructing. However, the project was canceled 
after unsuccessful tests and a new platform, Compass Arrow, was built for 
reconnaissance missions in China.2° Overspending and time delays hurt Compass Arrow 
and it was canceled in 1974 as part of cost-cutting efforts after the Vietnam War. 
Furthermore, the demise of Compass Arrow put into question the very idea of unmanned 


reconnaissance.26 


During the 1980s, 


There was a significant shift in U.S. and NATO strategy for countering the 
Soviet threat to western Europe...Reliance on nuclear retaliation was seen 
as unsatisfactory solution...The new NATO strateg[ic]...concept relied 
heavily on the development and use of new robotic types of weapons, such 
as RPVs...27 


22 B. C. Hacker, “The Machines of War: Military Technology in Twentieth Century Europe,” History 
and Technology 21:3 (2005), 274, as quoted in Krishnan, Killer Robots, 19. 


23 Ehrhard, Air Force UAVs, 11. 
24 Krishnan, Killer Robots, 21. 

25 Ehrhard, Air Force UAVs, 10. 
26 Ehrhard, Air Force UAVs, 11. 
27 Krishnan, Killer Robots, 23-24. 


“At the forefront of the technological development was DARPA [Defense Advanced 
Research Project Agency]... [whose] Strategic Computing Initiative (SCI)...aimed to 
achieve ‘real’ artificial intelligence within a decade.”28 The U.S. developed a new secret 
UAV program, the Advanced Airborne Reconnaissance System (AARS) to counter 
Soviet mobile nuclear ballistic missile systems. Their detection and tracking created a 
U.S. requirement for a “true loitering surveillance.”29 The AARS was designed to meet 
that need as they would provide the constant monitoring that satellites, U-2s, and SR-71s 
could not. One of the main components necessary for the mission was autonomy as well 
as a high altitude capability and intercontinental range from the U.S. However, when the 
Cold War ended, so did the program. 

Although SCI produced some tangible results in advances of computer 

technology, the ultimate aims of developing truly autonomous weapons 

were obviously not achieved. Nevertheless, a whole range of new robotic 

weapons were developed and fielded, most importantly the cruise missile, 


several automated air defense systems (Phalanx, Aegis, Patriot) and 
automated rocket artillery (MLRS).3° 


Several other UAV platforms were tested, but the end of the Cold War resulted in 
the termination of many of the UAV development programs. However, the threat in the 
Balkans during the 1990s led the Pentagon to take another look at some of the projects. 
Each service had its own black program in the 1980s, including the Navy’s Amber 
program, which indirectly led to the development of the Predator in the 1990s. One 
significant advantage of the Predator was that it could be controlled over satellites 
through a data link. New GPS technologies were also a tremendous asset that enabled the 


Predator system to evolve.3! 


Israeli-developed Pioneer drones were used for reconnaissance missions in the 
1991 Iraq War and Predators routinely flew over the Balkans in support of NATO 


operations in the 1990s. UAVs continued their reconnaissance missions in the more 





28 Krishnan, Killer Robots, 23-24. 
29 Erhard, Air Force UAVs, 13. 
30 Krishnan, Killer Robots, 24. 

31 Bhrhard, Air Force UAVs, 22. 
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recent wars in Iraq and Afghanistan, but also began firing hellfire missiles in support of 
combat operations. Their new role immediately improved the ability of Coalition Forces 
to engage specific threats, “but also [improved] other functions such as clearing the way 
for the attack with manned systems.”32 The resort to lethal force in combination with 
RPAs, however, raised new legal and ethical considerations, including charges of war 
crimes.33 The military continues to rely heavily on the Predator as well as a new RPA, 
the MQ-9 Reaper. In addition to their continued reconnaissance mission, the military now 


uses them to fire weapons, particularly Hellfire missiles. 


In April 2010, the Navy’s Fire Scout completed its first operational deployment.+4 
Designed to counter small swarming boats, the Fire Scout autonomously performs 
vertical take offs and landings and was the first remotely piloted vehicle (RPV) to fly off 
a ship on its own.*> In February 2011, the Navy also completed the first test flight of the 
X-47B, a bat-winged unmanned jet with stealth characteristics3° The advent of the 
autonomous technology that both the Fire Scout and X-47B use coupled with the current 
offensive use of the Predators and Reapers is evidence of the possibility that future 
platforms may combine both autonomy and lethal force. The Navy may have a unique 


ability to employ future unmanned aircraft for strategic force multiplication. 


A 2008 report by the Center for Strategic and Budgetary Assessments, a 
Washington think tank that studies military technology, said that long- 
range pilotless aircraft had potential to transform carriers “from a power- 
projection system with outstanding global mobility but relatively limited 
tactical reach and persistence into a key component of a global 
surveillance-strike network.’ 37 


32 Krishnan, Killer Robots, 27. 


33 Philip Alston, “Report of the Special Rapporteur on Extrajudicial, Summary or Arbitrary 
Executions,” United Nations General Assembly, May 28, 2010, 
http://www2?2.ohchr.org/english/bodies/hrcouncil/docs/14session/A.HRC. 14.24.Add6.pdf. 


34 Rique Sanchez, “Unmanned Aerial Vehicles & Ethics,” Proceedings 136, 6 (2010): 66-69, 
http://proquest.umi.com.libproxy.nps.edu/pqdweb?RQT=3 | 8&pmid=2855 1 &TS=1294803434&clientId=1 
1969&VInst=PROD& VName=PQD&VType=PQD. 


35 Ronald Arkin, Governing Lethal Behavior in Autonomous Robots (Boca Raton: CRC Press, 2009), 
23, doi: 10.1201/9781420085952. 


36 Nathan Hodge, “Drone Will Call Aircraft Carriers Home,” The Wall Street Journal, February 8, 
2011, 7. 


37 Hodge, “Drone Will Call,” 7. 
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Similarly, Noel Sharkey references the Committee on Autonomous Vehicles in 
Support of Naval Operations’ 2005 report that stated: 

The Navy and Marine Corps should aggressively exploit the considerable 

war-fighting benefits offered by autonomous vehicles (AVs) by acquiring 

operational experience with current systems and using lessons learned 

from that experience to develop future AV technologies, operational 

requirements, and systems concepts.28 

The Navy has been crucial in the initial deployment of force in America’s recent 
wars. The Navy’s worldwide presence, on land and at sea provides the Armed Forces 
immediate strike capabilities that cannot always be matched by the other services. 
Political considerations heavily influence where the U.S. may deploy its ground and air 
forces, therefore, the Navy may be relied on for power projection abilities in future 


unmanned or autonomous platforms. 
B. MOVING FORWARD WITH UNMANNED PLATFORMS 


Today, as never before, the U.S. Armed Forces and congress are planning for a 
future force that relies heavily on unmanned aircraft. The Air Force’s proposed budget in 
2010 called for thirty-six percent of its planes to be unmanned platforms.3? The DoD’s 
Unmanned Systems Roadmap plans to aggressively develop UAVs and utilize them. 


Section 2.2 states: 


In Section 220 of the Floyd D. Spence National Defense Authorization 
Act for Fiscal Year (FY) 2001 (Public Law 106-398), Congress Stated two 
key, overall goals for the DoD with respect to UAS and UGV 
development: 


By 2010, one third of the aircraft in the operational deep strike force 
should be unmanned and 


38 Committee on Autonomous Vehicles in Support of Naval Operations National Research Council, 
Autonomous Vehicles in Support of Naval Operations, Washington DC: The National Academies Press, 
2005, as referenced by Noel Sharkey, “Saying ‘No!’ to Lethal Autonomous Targeting’”, Journal of 
Military Ethics 9: 4 (2010): 376. 


39 Siobhan Gorman, Yochi J. Dreazen, and August Cole, “Insurgents Hack U.S. Drones,” The Wall 
Street Journal, December 17, 2009, http://online.wsj.com/article/SB 12610224788909501 1.html. 


12 


By 2015, one third of the Army’s Future Combat Systems (FCS) 
operational ground combat vehicles should be unmanned.49 


Since this 2001 expression of Congressional intent to advance the 
introduction of unmanned systems into the Joint Forces, the DoD has 
taken positive steps toward achieving those goals...4! 


Ronald Arkin points out how rare it is that the DoD conducts “this sort of truly long-term 
planning in operation in any area, and it speaks to the commitment the DoD has made to 
this technology.’’42 He points out the importance of the deputy director of Unmanned 
Aircraft Systems’ comments that artificial intelligence may one day be used to operate 
autonomous aerial platforms against enemies and recognizes that the DoD roadmap plans 
for such a future in unmanned systems.*3 

An expert on military technologies, James Canton, at the Institute for 

Global Futures stated that ‘autonomy, even for armed robots is coming,’ 


including a machine that will hunt, identify, authenticate, and possibly kill 
a target without a human in the decision loop.** 


C. JOINT AIR DOCTRINE AND UAVS 


The U.S. military currently authorizes Joint Force Commanders to use any 
unmanned aircraft available to them in order to meet the demands of their missions.*> In 
fact, Joint Publication 3-52, the Doctrine for Joint Airspace Control, advocates exploiting 
the endurance of UAVS: “Most larger UA (Unmanned Aircraft) have considerably longer 


endurance times than comparable manned systems. Planners must exploit this capability 





40 The Army’s Future Combat Sytems (FCS) program was canceled in May 2009 as part of larger 
DoD budgetary cuts. This is may also be evidence that technology changes more quickly than the plans to 
implement it. 


41 DoD (Department of Defense), Unmanned Systems Roadmap 2007-2032, December 10, 2007, 6. 
42 Arkin, Governing Lethal Behavior, 8. 


43 5. Magnuson, “Robo Soldiers,” National Defense September (2007): 36-40, as referenced by 
Arkin, Governing Lethal Behavior, 8-9. 


44 Magnuson, “Robo Soldiers,” as quoted by Arkin, Governing Lethal Behavior, 8-9. 


45 Joint Publication 3-30, Command and Control of Joint Air Operations, (January 2010), I-34, 
http://www.dtic.mil/doctrine/new_pubs/jp3_30.pdf. 
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when tasking UA assets.’’4© The chain of command for the authorization to deploy lethal 
force from a remotely piloted aircraft involves a combination of “eyes on target” from the 
pilot in the U.S., a regional area commander, and a local commander on the ground in 
theater that ultimately authorizes the pilot to fire. Humans are still very engaged in the 


control loop. 


There is not much difference in air space doctrine regarding unmanned platforms 
compared to doctrine that addresses traditional platforms. Joint Publication 3-30 states 


the following mission and flight planning considerations: 


Current doctrinal planning considerations for manned aircraft are 
applicable to UA, with minor modification. 


UA flights must follow all approved planning, guidance, and procedures 
as prescribed in the AOD, ACO, ATO, and SPINS.*7 


The doctrine highlights the importance of defensive and friendly fire considerations with 


UAVs as well. 


Our adversaries are developing and acquiring UASs, so it is imperative 
our C2 and DCA nodes are able to differentiate between friendly and 
enemy UAs and cruise missiles. ACPs must not allow a window of 
opportunity for adversaries to exploit. Specifically, the use of coordinating 
altitudes and standard use Army aircraft flight routes by UASs enables 
efficient and timely use of the airspace, but also makes it more difficult for 
air defense operators to differentiate between friend and foe. This type of 
airspace control is typically procedural control, and not positive control. 
Therefore, UAS operators must follow prescribed airspace control 
procedures and air defense identification procedures in order to prevent 
fratricide and/or allow enemy UAS exploitation of that airspace. 
Additionally, the use of coordinating altitude penetrating restricted 
operations zones (ROZs) to separate UASs from other airspace should be 
kept to a minimum.*8 





46 Joint Publication 3-52, Joint Airspace Control, (May 2010), I-11, 
http://www.dtic.mil/doctrine/new_pubs/jp3_52.pdf. 


47 Joint Publication 3-30, INI-34-35. 
48 Joint Publication 3-30, III-35. 
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Regarding the targeting by unmanned aerial vehicles with lethal force, 

Joint Publication 3-30 provides the following instruction: 
UAS C2 for Dynamic Targeting. Recent operations have demonstrated 
that UASs can be critical to the success of dynamic targeting missions and 
its prosecution of targets of opportunity (unplanned, unanticipated) or 
TSTs. Commanders of UASs should follow established procedures for 
executing dynamic targeting operations. Dynamic targeting situations may 
require UASs to support CAS, strike coordination and reconnaissance, air 
interdictions, other joint fires missions, and PR. Specific tasks for the UAS 
may include: target acquisition/marking, terminal guidance of ordnance, 
providing precision coordinates for Global Positioning System (GPS) 
aided munitions, delivery of onboard precision-guided ordnance, tactical 
assessment, BDA, and retargeting (i.e., “shoot-look-shoot’). In the 
dynamic targeting role, UASs are routed, controlled, and deconflicted in 
the same manner as fixed- and rotary-winged manned aircraft, as outlined 
in joint doctrine.*9 


The doctrine highlights the importance of targeting decisions, but it is not exactly clear 
what the rules of engagement (ROE) are in all targeting situations for unmanned aircraft 


from the prose in this doctrine. 
D. DILEMMAS WITH CURRENT UCAVS 


Already, the deployment of UCAVs has raised questions that the Laws of Armed 
Conflict do not address. Current deployment poses a range of dilemmas in law and for the 
military’s sense of sense of professional ethics. These dilemmas are largely associated 
with discrimination and noncombatant injury in conjunction with otherwise-permissible 
targeting inside established zones of combat, and projection of force outside established 
zones of combat.5° John Markoff reports, “predators have generated international 
opposition and prompted accusations of war crimes” due to collateral damage and 


noncombatant deaths in Iraq and Afghanistan.>! The introduction of autonomy into the 





49 Joint Publication 3-30, III-34. 
50 George R. Lucas, Jr. assisted in developing the proposal for this thesis. 


51 John Markoff, “War Machines: Recruiting Robots for Combat,” New York Times, November 27, 
2010, http://www.nytimes.com/2010/1 1/28/science/28robot.html. 
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present unmanned platforms will open new lines of inquiry with respect to governance, 


accountability, permissible and ethical use of autonomous platforms. 


It is prudent and wise for the military to ask critical questions now regarding the 
evolving nature of warfare and to decide the best future courses of actions with the new 
technology. The solution is not to determine regulations that will cover every possible 
situation in which an autonomous UCAV might find itself fighting. Rather, a more 
prudent action is to seek whether the environment exists or can exist where UCAVs can 
make strike decisions in keeping with the LOAC while also remaining in compliance 


with the professional military ethic and the Just War Tradition. 
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Hl. AUTONOMY 


While ethical concerns exist with the current deployment of UCAVs, a whole new 
set of questions accompanies the prospect of autonomy. For the purpose of analysis in 
this research, “autonomy” will be discussed in reference to a UCAV that makes decisions 
on its own to deploy lethal force.52 James Drennan argues that, “robotics can have a 
game-changing impact on par with guided munitions or nuclear weapons.”>3 These 
benefits were also highlighted in the Executive Summary and Command Brief for the 
10th Annual McCain Conference on Military Ethics and Leadership: 

Present unmanned systems reduce the risk to our war-fighters by 

providing a sophisticated stand-off capability that supports intelligence, 

command and control, targeting and weapons delivery. These systems 

also improve situational awareness and reduce many of the emotional 

hazards inherent in air and ground combat, thus decreasing the likelihood 

of causing civilian noncombatant casualties. Autonomous versions of 

these unmanned systems would sense, decide, and act more quickly than 


humans, bolster conventional deterrence in anti-access environments, and 
further reduce personnel costs.°* 


Computer scientist Ron Arkin points out a recent U.S. Army Solicitation for 
Proposals that emphasizes the need for systems to operate autonomously and have the 
ability to engage hostile targets, but leave the final strike decisions to human control.55 
However, the solicitation also proposes that, “Fully autonomous engagement without 
human intervention should also be considered, under user-defined conditions, as should 


both lethal and nonlethal engagement and effects delivery means.”°° The proposal for 





52 Bradley Jay Strawser, “Moral Predators: The Duty to Employ Uninhabited Aerial Vehicles,” 
Journal of Military Ethics 9 (2010): 349. 


53 James E. Drennan, “How to Fight an Unmanned War,” Proceedings 136 (2010): 58-63, 
http://www.usni.org/magazines/proceedings/2010- 1 1/how-fight-unmanned-war. 


54 Edward Barrett, “Executive Summary and Command Brief,” Journal of Military Ethics 9 (2010): 
426, doi: 0.1080/15027570.2010.540890. 


55 Arkin, Governing Lethal Behavior, 26-27. Italics added. 
56 Arkin, Governing Lethal Behavior, 27. Italics added. 
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increased autonomy and the possibility of fully autonomous lethal platforms demands 


introspection under the Laws of Armed Conflict, as well as the Just War Tradition. 


A. LAWS OF ARMED CONFLICT 


The laws of war or the LOAC are a combination of Arms Control and Law of 
War Treaties and Protocols, customs in profession of arms, and the Just War Tradition. 
The modern LOAC were originally developed from the Hague Conventions in 1899 and 
1907, but following World War II, “sixty-four nations gathered in Geneva in 1949 to 
reach a decision on expanding the canon of the international Laws of War.’’>7 

The law of war encompasses all international law for the conduct of 

hostilities binding on the United States or its individual citizens, including 


treaties and international agreements to which the United States is a party, 
and applicable customary international law.°® 


All members of the U.S. Armed Forces are instructed to comply at all times with the laws 


of war. Furthermore, it is DoD policy that: 


Members of the DoD Components comply with the law of war during all 
armed conflicts, however such conflicts are characterized, and in all other 
military operations. 


The law of war obligations of the United States are observed and enforced 
by the DoD Components and DoD contractors assigned to or 
accompanying deployed Armed Forces.>? 


The following are the Arms Control and Law of War Treaties and Protocols since 1954 


that the U.S. ratified and are a significant contribution to the Law of War Manual. 


The 1972 Convention on the Prohibition of the Development, Production, 
and Stockpiling of Bacteriological (Biological) and Toxin Weapons. 


57 John W. Bauer, “Justice: A Problem for Military Ethics during Irregular War,” (Monograph, School 
of Advanced Military Studies, U.S. Army Command and General Staff College, 2008), 16. 


58 Department of Defense Directive, Number 2311.01E, (May 2006, change 1, November 2010), 
Section 3.1, http://cryptome.org/dodi/dodd-2311-01le.pdf. 


59 Department of Defense Directive, Section 4. 
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The 1976 Convention on the Prohibition of Military or any other Hostile 
Use of Environmental Modification Methods. 


The 1980 Convention on Certain Conventional Weapons and its protocols 
I (nondetectable fragments), II (landmines, booby traps, and other 
devices), and II (incendiary weapons). 


The 1993 Convention on the Prohibition of the Development, Production, 
Stockpiling and Use of Chemical Weapons and on Their Destruction. 


The 1995 Protocol IV (Blinding Laser Weapons) to the 1980 Convention 
on Certain Conventional Weapons. 


The 1996 Amended Protocol II to the 1980 Convention on Certain 
Conventional Weapons. 


The 2005 Additional Protocol III to the 1949 Geneva Conventions. 


The 2006 Protocol V (Explosive Remnants of War) to the 1980 
Convention on Certain Conventional Weapons. 


As the U.S. is signatory to all of these treaties, under provisions of the U.S. Constitution, 


they are the law of the land as well. ©! 


The DoD’s Unmanned System’s Roadmap also recognizes the following arms 


control agreements that address the use of unmanned systems: 


U.S. Government arms control agreements concerning unmanned systems 
include the Wassenaar Arrangement (WA), the Missile Technology 
Control Regime (MTCR), the Treaty on Conventional Armed Forces in 
Europe (CFE), the Vienna Document 1999 (VDOC), the Intermediate- 
Range Nuclear Forces Treaty (INF), the Global Exchange of Military 
Information (GEMIJ), and the United Nations Transparency in Armaments 
Resolution (UNTIA). Conventional arms agreements that do not name 
unmanned systems, but mention military air and ground vehicles include 


60 w. Hays Parks, “‘National Security Law in Practice: The Department of Defense Law of War 
Manual,” 2010, http://www.abanet.org/natsecurity/hays_parks_speech11082010.pdf. 


61 “This Constitution, and the Laws of the United States which shall be made in pursuance thereof; and 
all treaties made, or which shall be made, under the authority of the United States, shall be the supreme law 
of the land; and the judges in every state shall be bound thereby, anything in the constitution or laws of any 
state to the contrary notwithstanding.” U.S. Constitution, article VI, paragraph 2, “Supremacy Clause.” 
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the CFE, VDOC, INF, GEMI, and UNTIA. Conventional arms agreements 
that address unmanned systems directly include the WA and MTCR.® 


Ron Arkin references the Unmanned Systems Safety Guide for DOD Acquisition 
that proposes guidelines for their restraint: 
DSP-6: The UMS [unmanned system] shall be designed to prevent 


uncommanded fire and/or release of weapons or propagation and/ or 
radiation of hazardous energy. 


DSP-13: The UMS shall be designed to identify to the authorized entities 
the weapon being released or fired, but prior to weapon release or fire. 


DSP-15: The firing of weapon systems shall require a minimum of two 
independent and unique validated messages in the proper sequence from 
authorized entities, each of which shall be generated as a consequence of 
separate authorized entity action. Both messages should not originate 
within the UMS launching platform.% 


Current doctrine is intended to guide joint forces commanders as they seek to 
determine the best course of action to succeed in their missions.6+ Maris McCrabb 
emphasizes the need for air power doctrine to evolve to conform to new realities in war.®© 
McCrabb’s argument for updated doctrine is judicious in the present question of 
autonomy in UCAVs. As Wayne Hughes recognizes, “such a doctrine will involve 
constant revision to link new technologies with new tactics, and to integrate the 


geopolitical environment with American economic realities.’’© 





62 DoD, Unmanned Systems Roadmap 2007-2032, 64. 


63 DoD (Department of Defense), Unmanned Systems Safety Guide for DOD Acquisition, June 27, 
2007a. as referenced by Arkin, Governing Lethal Behavior, 27. Italics added. 


64 Joint Publication 3-30, II-34. 


65 Maris McCrabb, “The Evolution of NATO Air Doctrine,” in The Paths of Heaven: The Evolution of 
Airpower Theory, ed. Phillip S. Meilinger, (Maxwell Air Force Base: Air University Press, 1997), 443. 


66 Wayne P. Hughes Jr., “Robot Ethics and Future War,” (working paper, Operations Research 
Department, Naval Postgraduate School, Monterey, CA, 2011). 
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B. ETHICAL CONSIDERATIONS 


Ethical challenges also need to be considered as the military develops evolving air 
power doctrine. The development of policy that will address autonomous capabilities in 
UCAVs cannot confuse what is legal with the broader demands of morality and military 
professionalism.®’ As Krishnan states: 

There is a fine, but important, difference between what is legal and what is 

moral in war - a difference that is sometimes overlooked in discussions on 

military ethics, which tend to focus more on the legality aspect. This can 

be problematic. Wars and actions in war might adhere to existing legal 

standards. At the same time, they could violate moral standards that lack 

legal codification. Besides, not all laws are in perfect harmony with 

common perceptions of what is moral and this can result in situations 


where law and morals conflict. One and the same action may be perfectly 
legal, but may also be perceived as being highly immoral. 


There are two levels of effective ethical arguments for and against the introduction of 
autonomy in UCAVs. There are challenges and obstacles to the ethical use of 
autonomous UCAVs, but there are also benefits to be derived. Both need to be 
considered, including (but not limited solely to) economic factors. Ethical considerations 


in this research will be considered under the framework of the Just War Tradition. 
1. Just War Tradition 


The Just War Tradition, stemming from philosophical reflections on warfare by 
Augustine and Thomas Aquinas, and codified by Hugo Grotius in the seventeenth 
century, is the foundation for the Laws of War. “Just war theory insists on a fundamental 
moral consistency between means and ends with regard to wartime behavior.”©? While 
reminding the military that jus in bello (justice in war) cannot be confused with an 


itemized checklist, Brian Orend says the principle shares common values with the other 


67 Thomas B. Grassey, “Why Ethics Is So Hard,” in Ethics and the Military Profession Third Edition, 
ed. George R. Lucas, Jr. and W. Rick Rubel (Boston: Pearson, 2010), 18. 


68 Krishnan, Killer Robots, 117. 


69 Brian Orend, “Jus in bello - Just Conduct in War,” in Ethics and the Military Profession Third 
Edition, eds. George R. Lucas, Jr. and W. Rick Rubel (Boston: Pearson, 2010), 273. 
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two categories, jus ad bellum (just war) and jus post bellum (justice after war), that make 
up the just war theory. These include rejecting aggression, restraining warfare, and 
protecting the state rights of legitimate communities as well as individual human rights.”° 
Jus in bello guides the military’s conduct in war. Whereas jus ad bellum refers to a state’ s 
decision to fight wars, jus in bello deals more specifically with “anyone involved in 
formulating and executing military strategy during wartime.”’! Jus in bello includes four 


basic principles: 


e Proportionality 

® Military necessity 

e Discrimination 

e Prevention of unnecessary suffering ’2 


The Laws of War discussed in the previous section certainly do not address the 
infinite number of legal and ethical questions that will arise if autonomous UCAVs are 
flown above future battlefields. It is necessary, instead, to look back to the original 
meaning of jus in bello to guide our Armed Forces in the increased reliance on artificial 


intelligence. 


In a monograph written for the Army’s School of Advanced Military Studies, 
John Bauer argues that, “instead of encouraging ethical reflection, Just War Tradition has 
today caused military ethics to drift toward legalism.”’3 The answer to the ethical 
questions of autonomy and lethality cannot adequately be addressed solely through legal 
analysis. Bauer points out there is a conflicting dichotomy between Aristotelian virtue 
ethics and Kantian procedural ethics when understood in the context of the Just War 
Tradition. “Few can deny the need for both approaches to ethics: one that emphasizes the 


formation of the virtuous man and the other that emphasizes moral rules that can directly 


70 Orend, “Jus in bello,” 273. 
71 Orend, “Jus in bello,” 274. 


72 Derek I. Grimes, John Rawcliffe, and Jeannine Smith, eds. Operational Law Handbook 
(Charlottesville, Virginia: The Judge Advocate General’s Legal Center and School, 2006), 12-14, in Bauer, 
“Justice: A Problem for Military Ethics,” 17. 


73 Bauer, “Justice: A Problem for Military Ethics,” iii. 
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inform ethical decisions.”’4 While neither viewed justice as one-dimensional, their two 
legacies can appear fundamentally opposed on the surface. Aristotle stressed virtue as the 
basis of ethics while Kant valued formalism.’> In his monograph, Bauer argues that there 
is an ethical void that exists between these two competing notions that opens the door to 
an interpretation that “goes so far as to question the need for any ethics whatsoever, to 


further doubt the existence of any universal meaning of justice.’’76 


The ethical void and confusion over the applications of the Just War theory are 
inherent in the present question. The legal questions in warfare are necessary as the 
military considers the introduction of autonomous UCAVs into its force structure. They 
are not sufficient, however, to adequately address the magnitude and scope of the change, 
which also impacts military professionalism and ethics. The introduction of autonomy 
presents an array of new challenges in modern warfare that is increasingly urban and 
involves nonstate actors. Laws cannot be written for every possible challenge in war. An 
applied understanding of the Just War theory will help guide the use of lethal force in 
unmanned platforms and should be given careful consideration before the introduction of 


autonomy in lethal decision making. 


There are two opposed tendencies in the future force structure considerations for 
our Armed Forces: Should more humans be put in the loop or should there be less? On 
the one hand, the Armed Forces have recently put more humans on the ground to achieve 
much-needed cultural sensitivity in what is termed “the human terrain.”7”7 On the other 
hand, however, introduction of autonomous unmanned platforms will put humans at an 
even greater distance from that terrain. Tactical decisions regarding the deployment of 
autonomous UCAVs in battle will need to consider how the receiving parties will react to 
the robotic projection of force and how that will affect mission accomplishment. The 


overall military strategy must determine how to best resolve these conflicting tendencies. 


74 Bauer, “Justice: A Problem for Military Ethics,” 20. 
75 Bauer, “Justice: A Problem for Military Ethics,” 20. 


76 William Frankena, Ethics, 2nd Edition (Upper Saddle River: Prentice Hall, 1973), 110 as quoted in 
Bauer, “Justice: A Problem for Military Ethics,” 20. 


77 George Lucas, Anthropologists in Arms: The Ethics of Military Anthropology (Critical Issues in 
Anhropology), (Lanham, MD: AltaMira Press, 2009). 
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Some of the major ethical challenges related to the introduction of autonomy in 


unmanned combat aerial vehicles include: 


e Attribution and accountability for mistakes and war crimes. 
e Protecting the lives of American soldiers 

e Mission accomplishment and cost effectiveness. 

e Discrimination and proportionality 

e Proliferation and theft. 

e The threshold for the risk of war and last resort. 


2: Attribution 


Attribution is addressed separately from the other above-mentioned ethical 
challenges because it does not necessarily have “pros” and “cons” associated with the 
question. The other challenges listed above will be addressed separately and divided into 
two categories, either on the positive side of ethical considerations or on the negative 


side. 


The deployment of autonomous UCAVs “could potentially interrupt the clear 
chain of military command that is required by international law.”’8 The UCAV could “be 
both capable of autonomous behavior and incapable of assuming responsibility for its 
actions.”’? Some proponents of the introduction of autonomous platforms dismiss 
concerns that may be associated with their lethal use by arguing that as long as someone 
can be held responsible for the “killing,” then it is in keeping with the Just War Tradition 
and the LOAC. Some argue that the company that designed the autonomous agent should 
be held responsible if something goes awry with the mission and innocent people are 
inadvertently killed. Another proposed legal solution may be to institute “no fault 


liability,” in order to encourage innovation and development and exonerate developers 


78 Krishnan, Killer Robots, 103. 
79 Krishnan, Killer Robots, 103. 
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from culpability.80 But this inherently raises ethical questions that cannot be easily 
dismissed. Similar to cyber warfare, the U.S. Armed Forces faces an attribution problem, 
because if we do not know who did something, we do not know who to punish for it. The 
simple solution proposed of holding the last human in the “kill chain” responsible for the 
actions of the autonomous UCAV is not sufficient to address the ethical challenges 


associated with the introduction of autonomous platforms. 


George Lucas summarizes Australian philosopher Robert Sparrow’s original 
objections regarding the (un) accountability of autonomous platforms: 

One is prohibited under existing international law from proposing or 

developing any weapons system for whose use or misuse military personnel 

or their governments cannot be held reasonably accountable under LOAC. It 


is difficult, if not impossible, however, to conceive of how an autonomous 
lethal machine can be meaningfully held accountable for its actions.®! 


With the introduction of autonomy in UCAVs, someone may still be required to 
“press go” for a mission, but these platforms may also respond defensively to certain stimuli. 
This is similar to “smart” weapons today as the nature of an autonomous platform is one that 
might decide for itself when to fire and ideally adapts to the real time environment and 
operates within the LOAC. The sole reliance on artificial intelligence to decide “when” and 
“where” to strike could absolve the human in the loop of some responsibility. Attribution is a 
valid question with regard to the introduction of autonomous platforms, but assigning 
accountability to the UCAV operator or squadron commander is not sufficient to address the 


full scope of ethical considerations. 


3. Arguments for the Introduction of Autonomy 


The remaining moral challenges outlined above, unlike attribution, figure into 
arguments both for and against the introduction of autonomy. The following ethical 
considerations have been advanced in favor of the introduction of autonomy in unmanned 


systems. 


80 Ryan Calo, “Open Robotics,” presentation at Stanford University, November 11, 2010. Presentation 
on a forthcoming paper by Maryland Law Review. 


81 Robert Sparrow, “Killer Robots,” Journal of Applied Philosophy, 24:1, 2007, 62-77 as referenced 
by George Lucas Jr., “Postmodern War,” Journal of Military Ethics 9 (2010): 298, 
doi:10.1080/15027570.2010.536399. 
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a. Protecting the Lives of American Soldiers 


If autonomous UCAVs can save the lives of American soldiers, it may be 
more ethical to use them instead of manned platforms. Anthony Hartle recognizes that, 
“the U.S. government has concluded that recognition of the human rights of enemy 
combatants does not require or justify placing American lives at risk.” 82 Already 
recognized with the use of remotely piloted aircraft by Bradley Strawser, ethical 
considerations for saving human life include the growing lethality of high-intensity war, 
enemies bound by no rules, exposure to environmental dangers, and the increasing 
psychological costs of combat are all factors that encourage the use of autonomous 
platforms.83 The question of whether it is unethical not to deploy autonomous aircraft if it 
will save American soldiers’ lives is not new. During World War II, in reference to 
deploying Weary Willies, B-17s and B-24s packed only with explosives, the USAAF 
Chief, Gen. H. H. Arnold, claimed, “If you can get mechanical machines to do this, you 


are saving lives at the outset.”°4 


While the deployment of autonomous UCAVs may save the lives of 
American soldiers, more innocent noncombatant lives may be lost. Hartle comments on 
this challenge to the Armed Forces: 
Difficult choices result from a conflict between the American commitment 


to freedom and respect for individual persons, on the one hand, and our 
right to defend ourselves and enhance our own security, on the other. 85 


The nature of future conflicts will also influence what kinds of risks both American 
soldiers and civilian noncombatants will face. In contrast to the past, the promise of 


precision targeting of autonomous unmanned platforms offer hope. 


82 Anthony E. Hartle, Moral Decision Making in Military Decision Making (Lawrence: University 
Press of Kansas, 2004), 147. 


83 Strawser, “Moral Predators,” 342-368, and Krishnan, Killer Robots, 117. 


84 Rebecca Grant, preface to Air Force UAVs: The Secret History, by Thomas P. Ehrhard (Arlington: 
Mitchell Institute Press, 2010), 2. 


85 Hartle, Moral Decision Making, 147. 
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b. Mission Accomplishment and Cost Effectiveness 


Richard L. Kugler reminds the military that “strategy comes with a dollar 
sign.”86 Similar to the argument to save American lives, the question of autonomy in 
UCAVs presents the military with a pointed question: should it not use the most effective 
means possible to accomplish its mission? John Canning argues that “leaving a man in 
the loop would be ‘a performance and cost killer’ when considering the employment of 


large numbers of unmanned systems.”’8/ 


Ron Arkin advocates a future force that combines autonomous systems 
with humans working together on the battlefield, in which the autonomous systems will 
act as a force multiplier.88 If autonomous UCAVs are more cost-effective at mission 
accomplishment than manned aircraft, it may be unethical not to use them.89 Removing 
the human from the loop may actually enhance the timely decision-making capability 


necessary to deploy lethal force. 


The introduction of autonomous capability in UCAVs may also serve as a 
deterring threat to America’s enemies. Recognizing the lethal force America may deploy 
at little risk to U.S. military lives may influence the perceptions of conventional foes and 


nonstate actors about America’s will to fight. James Drennan states: 


Enemies of the United States have learned over the past few decades that 
sapping the American public’s will to fight can be an effective strategy. 
But if they are faced with an assault that effectively eliminates the hazard 
to American soldiers and innocent civilians, enemies will surely question 
that strategy.99 


86 Richard L. Kugler, Commitment to Purpose (Santa Monica: RAND, 1993) as quoted in McCrabb, 
“The Evolution of NATO Air Doctrine,” 444. 


87 John Canning, “Concept of Operations for Autonomous Systems,” website (updated 2006), 
http://www.dtic.mil.ndia/2006disruptive_tech/canning.pdf as quoted in Krishnan, Killer Robots, 106. 


88 Arkin, Governing Lethal Behavior, 9. 
89 Krishnan, Killer Robots, 119. 
90 Drennan, “How to Fight,” 58-63. 
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This may serve a morally good purpose if it can force other nations to negotiate with, 
instead of fighting against, American forces. We may not soon forget the Iraqi soldiers 


who surrendered to Pioneer UAVs in 1991.9! 


There is a potential for autonomous platforms to be more mission effective 
as force multipliers and also more cost effective in terms of manpower needed than 
traditional platforms.9* Krishnan argues that “autonomous systems would require no 
human operators and could be made much smaller than manned systems, or even tele- 
operated systems, thus reducing the numbers of required maintenance technicians.”9 
Additionally, in the future, their increased reliance could reduce the need to maintain 


such a large military.°* 
c. Discrimination and Proportionality 


The potential ability for autonomous UCAVs to discriminate is used as an 
argument both for and against their ethical deployment. Elbridge Colby recognizes, “for 
the use of force to be morally tolerable it must be discriminate - civilians may not be the 
object of direct, deliberate attack - and it must be proportionate to the evil confronted and 
the good achieved.”?> While autonomous UCAVs may one day discriminate better than a 
human, there is the potential for violation of the principles of discrimination and 


proportionality. 


The use of autonomous lethal force has the potential of “humanizing 
warfare.”97 Proponents of autonomy and lethal force point out the vices of human 


soldiers’ behavior in warfare, particularly in examples where soldiers react inadvertently 


°l Ted Shelsby, “Iraqi Soldiers Surrender to AAI’s Drones,” The Baltimore Sun, March 2, 1991, 
http://articles.baltimoresun.com/1991-03-02/business/1991061100_1_rpv-aai-drones. 


92 The proposition for less manpower has been challenged because currently as over hundred fifty 
personnel can be required to support a single currently deployed Predator. 


93 Krishnan, Killer Robots, 37. 

94 Krishnan, Killer Robots, 35. 

95 Elbridge A. Colby, “Keeping the Peace,” First Things 209 (2011): 27. 
96 Sanchez, “Unmanned Aerial Vehicles,” 66-69. 

97 Krishnan, Killer Robots, 145. 
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based on emotions such as fear and anger. The possibility exists for autonomous 
platforms to employ lethal force only in a predetermined, prescribed, fashion - lethal 
force that is not only unaffected by human emotion, but in full compliance with the 
mission goals and the LOAC. Wayne Hughes proposes the benefit of using more humane 
autonomous systems: 
Missiles, unmanned vehicles, and robots are cold blooded. The ethicist 
emphasizes their detached cruelty as a vice. I emphasize their coolness 
under fire as a virtue. A robot won’t panic, or duck, or flee, or lose its 
temper. Analyzing robot warfare will be easier than analyzing combat 
between humans when mind and spirit are prominent. The purpose of 


gunfire may be as much to destroy an enemy’s morale or to keep their 
heads down as it is to kill them.?8 


The arguments for autonomous platforms exhibiting more humanity 
presuppose the development of technology that will accurately distinguish between 
enemies and noncombatants. This may not be technically feasible in the near term. For 
example, we currently have the capability to recognize and target the shape of an AK-47 
rifle. Even though the AK-47 may be identified, however, we cannot simultaneously 
determine who it is who is carrying that weapon. An individual carrying an AK-47 might 
be an enemy combatant, or merely harmless shepherd, guarding his flock in a war zone. 
The ability of autonomous platform to “discern between legitimate and illegitimate 
targets (such as civilian bystanders, or injured or surrendering enemy combatants), and to 
subsequently apply the requisite legal and moral principles of military necessity and 
proportionality” was cited as an area of concern in the Executive Summary of the 2010 


McCain Conference.”? 
4. Arguments Against the Introduction of Autonomy 


The following ethical considerations have been used to argue against the 


introduction of autonomy in unmanned systems. 
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a. The Threshold for the Risk of War and Last Resort 


The introduction of autonomy in unmanned systems may not only increase 
the risk of war, but it may also challenge the very notion of nations only engaging in war 
as an option of last resort because it lowers the threshold of going to war. Hughes argues, 
“a simple policy of last resort for cyber war or robotic attacks is untenable. !°° This 
concern was also recognized at the 2010 McCain Conference: 

Reduced operating costs and the lower public profile associated with 
unmanned systems, as compared with conventional manned combat 
operations, may inadvertently lower the threshold for resorting to war, 


thereby undermining compliance with the traditional “just war” 
requirement that war only be declared as a last resort.!0! 


Furthermore, the increased use of autonomous platforms might make war 
itself less destructive and costly. This might make it easier to rationalize their 
employment in inter-state conflict and could lead to our nation resorting to war in 


situations that may be better resolved diplomatically .! 
b. Proliferation and Theft 


Autonomous technology may be intercepted or stolen by enemies and 
potential adversaries. It could be electronically hijacked in the sense that someone could 
intercept a live feed and take over command of the UCAV. In Iraq, Shiite militants have 
intercepted live video feeds from U.S. Predators.!93 Depending on what level of 
autonomy exists in the UCAV, a mission may or may not be able to be altered by a 
hyacker. Today’s concerns over the security of nuclear compounds in Pakistan represents 
a parallel concern to the kind of threat that America could face as other nations exploit 


autonomous weapons technology, particularly unstable nation states. David Sanger 


100 Hughes, “Robot Ethics.” 
101 Barrett, “Executive Summary,” 427. 


102 George Lucas, Jr., “Postmodern War,” Journal of Military Ethics 9 (2010): 298, 
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writes, “it’s not hard to envision a situation in which [a] state’s authority falls apart and 


you’re not sure who’s in control of the weapons, the nuclear labs, the materials.” !04 


Nonstate actors and rogue agents who hijack control of a UCAV may have 
the ability to strike at the U.S. anonymously. Krishnan observes, “robotic weapons could 
proliferate widely and even enable nonstate actors to carry out new kinds of terrorist 
attacks.” !05 The nature of autonomous weapons is unique in its ability to conceal the 
identity of the war-fighter. It increases the opportunity for nonstate actors to deploy lethal 
force on a large scale. A counterpoint in favor of complete autonomy, however, is that a 
UCAYV “that does not need to exchange much data with control stations would be much 


harder to hijack, which makes weapons autonomy a highly desirable feature.” !06 


Great care must be taken to ensure the technology does not fall into the 
wrong hands. To alleviate this threat, the following recommendations were offered at the 


2010 McCain Conference: 


Consistent with the 2009 McCain Conference recommendations 
[regarding control of the operations of] private military contractors, 
military members must [likewise] maintain control of all strike systems. 


Deterring the negligent use of autonomous strike systems will require not 
only design features that recommend and record user actions, but also 
clear performance standards, unbiased testing, rigorous training, and well- 
crafted liability laws. 


Given the technical challenges of identifying illegitimate targets and 
calculating necessity and proportionality in complex and even novel 
environments, extreme caution should govern actual deployment of 
autonomous strike systems. 107 


104 David E. Sanger, “Obama’s Worst Pakistan Nightmare,” New York Times, January 8, 2009, 1, 
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The increased use of UCAVs could also make control stations more 
vulnerable, possibly provoking attacks on American soil.!08 Enemies may target the 
homeland control stations, which “could become primary targets for the enemy.”! The 
U.S. could alleviate this threat by deploying autonomous UCAVs from alternate locations 
that are not on American soil (such as Navy ships). By doing so, American citizens could 
be safer from threats that may exist even today near locations such as Creech Air Force 
Base, where unmanned platforms are currently controlled, rendering the base itself and its 
personnel more liable to direct attack. Another threat may still exist as “control stations 
that are in theatre could be easily identified and located because of the amount of radio 


traffic that passes through them.”!!° 
Cc, TACTICAL ENVIRONMENTS 


An important consideration for the military is what types of warfare it anticipates 
in the future. Autonomous UCAVs may be deployed to a wide array of tactical 
environments. These will most likely range from conventional warfare to battlefields that 
more closely resemble today’s counterinsurgency (COIN) operations. Within the range of 
possibilities, there are varying levels of consideration for air doctrine, and to which types 
of environments may present an opportunity for the introduction of autonomous lethal 
force. A short review of the taxonomy of these environments will help illustrate where 


the introduction of autonomous UCAVs may be morally permissible. 
1. Conventional 


A conventional battle, similar to the historic Battle of Midway, may present itself 
as an opportunity for the military to use autonomous platforms for needed force 
multiplication against a clearly defined enemy. In a sea battle, where fighting occurs 


primarily over open waters between clearly defined fighting forces, an environment could 


108 Drennan, “How to Fight,” 58-63. 
109 Krishnan, Killer Robots, 39. 
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exist where there is a high-degree of confidence in targeting. In this scenario, there is an 
implicit assumption that even though the enemy may not be physically distinguishable, 


two opposed forces are engaged with each other. 


Environments that mirror traditional battlefields where the U.S. military faces 
defined enemies may provide opportunities for the deployment of autonomous platforms. 
In more conventional combat zones, the air doctrine considerations may be more akin to 
those in existence or that address future manned platforms. The use of autonomous 
platforms with specified missions offers prospects for remaining in full compliance with 
the jus in bello principles of proportionality, military necessity, discrimination, and the 
prevention of unnecessary suffering. It will be important to achieve several metrics such 
as the certainty of the enemy and an assessment of the actual combatants and the 


potential for friendly fire. 


Future wars could also present the military with an environment similar to the 
first Gulf War in 1991 that was a striking example of what airpower can accomplish.!!! 
Philip Meilinger argues that coalition air operations closely followed General Douhet’s 
formula for victory—” gaining command of the air, neutralizing the enemy’s strategic 
‘vital centers,’ and maintaining the defensive on the ground, while taking the offensive in 
the air.”!!2 In a similar environment, there may a battle on land that is conventional with 
uniformed, state armies who do not make efforts to conceal or confuse their fighting 
forces with civilian inhabitants. This would conceivably make identification for 


autonomous UCAVs plausible. 


There could also be a much greater chance of collateral damage in a battle on 
land, even between traditional forces. The autonomous UCAV’s ability to discriminate 
may be dependent on how close enemies are to civilian populations. “If the goal of war is 


authentic peace...the greatest challenge to jus in bello in modern war is the preservation 
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of justice for civilian inhabitants subjected to war.”!!3 Positive identification of the 
enemy, minimizing collateral damage and friendly fire, and taking into consideration 
civilians that are unable to leave an environment should all factor in to the question of 


whether it may be ethical for the introduction of autonomous platforms. 


All wars will not directly imitate the model of air operations in the Gulf War. 
Future challenges may imitate those in the Balkans, Iraq and Afghanistan, or even 
Pakistan, where the U.S. did not declare war, but deploys lethal force via UCAVs within 
a governing allied nation’s borders. These nonconventional battlefields are thus more 


challenging from an ethical and legal perspective. 
2. Nonconventional 


The occupation of Iraq and the challenges faced by U.S. and allied soldiers 
could be indicative for the future of war and in line with the Pentagon’s 
expectation that urban operations will become more frequent and one of 
the main military challenges in the twenty-first century. While traditional 
warfare against conventional forces primarily fought on open terrain is 
possibly about to fade into history, unconventional and untraditional 
warfare against small nonstate forces like insurgents and terrorists hiding 
in cities within a civilian population is on the rise. More than half of the 
world’s current population of 6.7 billion lives in cities and about 4 percent 
in megacities with more than 10 million people...!!4 


Particularly in irregular warfare, ethical questions are not only relevant to the 
questions of autonomy in the unmanned lethal projection of force, but also crucial to 
mission success. DoD decisions on the appropriate use of autonomous systems must 
evaluate how America’s enemies and noncombatants in war will perceive them. The 
challenges of irregular warfare present a great challenge to the military professional ethic. 
Thomas Grassey argues, “two related factors... make ethics hard: the pressure of time, 
and the limits of knowledge.”!!5 American pilots and soldiers find themselves in 


environments that are confusing and rapidly changing, forcing them to adapt to all kinds 





113 Bauer, “Justice: A Problem for Military Ethics,” 19. 


114 United Nations, State of the World Population: Unleashing the Potential of Urban Growth (United 
Nations Population Fund, 2007), 9 as referenced by Krishnan, Killer Robots, 29. 


115 Grassey, “Why Ethics Is So Hard,” 16. 
34 


of battlefield conditions that are increasingly urban while culturally and politically 
sensitive. Hills states, “many military analysts claim that ‘urban operations are 
distinctive’ and that ‘urban environment’ is the most complex and challenging of 
environments in which armed forces have to fight.”!!© Bauer also recognizes the great 
challenges to jus in bello in irregular warfare: 

This has become the challenge our military faces as it engages in irregular 

warfare, where the lines between noncombatant and combatant become 

increasingly blurred and the problem of military ethics becomes even 

more tactically relevant.!!7 
While proponents of autonomy in unmanned platforms point out the possibility for 
digitized discrimination and precision, unconventional warfare challenges even the best- 
trained airmen and soldiers. America’s enemies regularly exploit its efforts in war to 
discriminate, often putting their own citizens’ lives at risk. The ethical considerations in 
nonconventional environments warrant further introspection. 

Winning over the civilian population is vital in today’s wars and is already an 
important consideration for the military’s current use of UCAVs. While the drone strikes 
in the Afghanistan and Pakistan are helping the U.S. successfully target al Qaeda and the 
Taliban, they are also challenging public perceptions of the American military in those 
countries as well. The deployment of autonomous UCAVs may exacerbate this dilemma. 
The challenges autonomous platforms will face in irregular warfare was also recognized 
at the 2010 McCain Conference: 

During counterinsurgency and stability operations (in which perceptions 

and relationships are crucial), local citizens may perceive these weapons 

as indiscriminate, indicative of cowardice, or evidence of the tepid 

commitments of allies. Additionally, ground robots are incapable of 


cultivating the personal relationships required to build trust and 
commitment necessary for mission success.!!8 
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Recent wars demonstrate the great difficulty, even for the best-trained forces, to 
successfully complete missions as they also work to win over civilian populations. Even 
if the weapon systems of the future do have the technological ability to distinguish 
between armed combatants based on physical recognition or weapon heat sensors or any 
other computerized recognition, there is a very real human element and response, 
particularly in urban warfare, that is often necessary to distinguish between friend and 
foe. The effort to make this distinction is consistent with the original purpose of the Just 


War Tradition, to achieve a lasting peace.!!9 
D. WEAPON CONSIDERATIONS 


Current weapon platforms, including the AGM-84 Harpoon missile, the AIM 120 
AMRAAM missile, the AEGIS and Phalanx systems and smart mines may further 
illustrate the potential for autonomy in UCAVs. The AIM-120 is an advanced, medium- 
range, air-to-air tactical missile that can be deployed from Navy and Air Force jets. It has 
a beyond-visual-range (BVR) capability and “fire and forget” ability in which the missile 
relies on its active radar to guide it to the target.!20 The MK 15 Phalanx Close-In 
Weapons System (CIWS) has autonomous “detect-through-engage” capabilities.!2! The 
anti-ship weapon, the HPM-84 Hapoon SLAM (Stand-Off Land Attack Missile) is also 
an over-the-horizon, fire and forget cruise missile that relies on active radar homing.!22 


Smart mines, developed for land and air forces, contain anti-armor and antipersonnel 
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36 


mine variants and operate autonomously.!23 The military has moved away from offensive 
“smart” mining, both for ethical reasons and the 1997 Ottawa Treaty, even though the 


US. is not a signatory to the treaty. !24 


As Krishnan recognizes, “sensors, mines and smart weapons...have at least some 
characteristics of robots: they are automated or programmed and they use sensors to 
direct them to their targets.”!25 The use of these systems demonstrates the need for 
caution before introducing autonomy in UCAVs. The use of the AIM-120’s BVR 
capability presumes, for example, that a civilian airliner is not in range of the missile. The 
Harpoon missile is programmed not to “turn on” until it is beyond any friendly forces.!26 
Similarly, the use of an autonomous UCAV would also need to consider what else might 
be in its range and assess the potential for error. In response to an Iraqi Silkworm missile, 
the Phalanx system on the USS Jarrett mistakenly struck the USS Missouri in the Persian 
Gulf in 1991.!27 The AEGIS system can operate autonomously, but it can also involve a 
human in the loop. The “automatic” response in CIWS can be detrimental, but keeping a 
human in the loop in semi-autonomous weapon systems can also be problematic. 
Tragically, in 1987, an Iraqi missile struck the USS Stark in the Persian Gulf. The missile 
could easily have been neutralized, automatically, by its Phalanx defense system, if it had 
been turned on.!28 Krishnan recognizes that “it might therefore be safer to automate a 


system completely than to allow too much human intervention.” !29 
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The examples of current weapon systems and capabilities to turn “on” or “off” 
automatic capabilities demonstrate that the role of the human in the loop matters in 
military ethics. The “role” of the human is dependent on many factors including 
peacetime and wartime rules and additionally what the current ROE are for particular 
geographic operational zones. Every decision cannot be predetermined for the ethical use 
of autonomous UCAVs or other autonomous features of weapon platforms. Certain 
decisions will always need to be left to area commanders, and the exact ROE should 
remain classified. This is necessary for adequate force protection. For example, if future 
UCAVs have autonomous capabilities that can be turned “on” or “off,” but there are 
public ROE about when this is permissible, enemy forces could exploit this knowledge in 
their favor. The military may decide to use the autonomous feature when there is not a 
risk of collateral damage and might appreciate the ability to enable the automatic option 


in the future event of an unforeseen major war. 


Two other factors that should be considered in autonomous UCAV platforms 
include latency factors and swarm control.!3° The importance of maintaining a human in 
the loop at some level may be necessary to optimize algorithms in UCAVs, but because 
of latency issues, the human operator may actually decrease the abilities for tasks such as 
landings and takeoffs. With the introduction of swarm control, everything could be 
integrated at the payload management level in UCAVs.!3! This could increase force 
multiplication. Missy Cummings, a systems engineer at MIT and former Navy fighter 
pilot, argues that the use of swarm control may decrease the need for a skills knowledge 
base for flying UCAVs, but lethal decision making may still very much rely on 


knowledge based skill reasoning. !32 
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IV. THE PROFESSIONAL MILITARY ETHIC 


The manner in which the nation’s military conducts itself as an instrument of 
American policy is grounded in the “professional military ethic,” reflecting the bedrock 
values of the nation and the time-honored traditions of the profession of arms. With the 
advent of autonomous lethal force in unmanned platforms, the answers to what actions 
provide the best military solutions to defeat our enemies, while exhibiting a respect for 


everyone caught up in war, can be confusing. 
A. HOW WE CONDUCT OUR WARS 


Anthony Hartle describes how the most pervasive societal values have affected 
the American professional military ethic.!33 America’s National Security Strategy in 
2002 and 2006 identified “[championing] aspirations for human dignity” as an essential 
task and listed it first both years, even above defeating terrorism.!34 Not only does 
championing human dignity mean the U.S. military must fight wars sometimes to 
empower those aspirations around the world, but in fact it must do so even in the midst of 


fighting those wars. 


More than ever before, the manner of fighting is a vital factor to America’s 
success in war and in achieving its national security strategy. Bauer describes how vital 


this concept is, particularly in the midst of irregular warfare or COIN. 


In irregular conflicts, just acts build legitimacy, win the support of the 
populous, undermine insurgencies, and bolster host nation governments. 
Injustices, on the other hand, fuel the ideological underpinnings of 
insurgencies and threaten American legitimacy in the eyes of the world. 
Since the attitudes of civilians are important to these factors, the central 
moral issue then becomes the just treatment of the civilian inhabitants 
affected by war. The international Laws of War were intended to work 


133 Hartle, Moral Decision Making, 134. 
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toward these ends. However, law by itself cannot achieve the full 
protection of civilians and ensure their just treatment. !35 


Hartle explains, “the American professional military ethic and the laws of war 
[should be] consistent.” !36 The Geneva Conventions were dedicated to the human rights 
of noncombatants in war and today the professional military ethic recognizes the inherent 
human dignity of individuals. However, a dichotomy often exists. Hartle claims, “the 
military professional must choose either to respect the rights of certain individuals or to 
achieve a particular objective that appears to be required by his commitment to serve 


society’s interests.” 137 
B. MORAL DISENGAGEMENT 


While there may be a use of autonomy in UCAVs that is consistent with the Just 
War Tradition and the Laws of Armed Conflict, there is a predicament stemming from 
the new distancing of the warfighter from the combat environment that should be 
considered. Krishnan observes, “military analysts have expressed their concerns with 
regard to the new strategy of casualty avoidance that is expressed in the growing 
emphasis on long distance warfare and robotics.” !38 Since the advent of the long-bow 
(one of the earliest man-portable indirect fire weapons), or more recently, the military’s 
reliance on air power, soldiers have been increasingly removed from the carnage of 
war.!39 Long-range missiles and other standoff weapons platforms present a similar 
dilemma. The distance from the killing in weapon systems used today removes the 
warfighter who presses the “kill” button from the actual killing, at least from the 


perspective of personal risk. 
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136 Hartle, Moral Decision Making, 147. 

137 Hartle, Moral Decision Making, 147. 

138 Krishnan, Killer Robots, 134. 
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David Grossman explains how the military strives to make it easier for the 
warfighter to kill from a distance that separates him from the actual identity of the target. 
The further the distance is from the actual killing, the easier it is for someone to 
psychologically “kill” his target. He describes how modern Navy sailors do not suffer the 
same negative psychological impacts as their fellow airman and soldiers do from killing. 
The reason “is that most of them don’t have to kill anyone directly, and no one is trying 
to specifically, personally, kill them.”!4° Grossman cites Gwynne Dyer: 

There has never been a similar resistance to killing among artilleryman or 

bomber crews or naval personnel...partly...this is due to the ‘same pressure 

that keeps machine-gun crews firing, but even more important is the 


intervention of distance and machinery between them and the enemy.’ 
They can simply ‘pretend they are not killing human beings.’ !4! 


The pilots of UCAVs may experience the same disconnect as the World War II “naval 
watriors [who] understood that they were killing humans just like themselves and that 


someone wanted to kill them, but emotionally they could deny it.” !42 


Cummings describes how jet fighter pilots tend to believe what they are told 
regarding strike decisions in stressful environments.!*3 The effect is described as 
“automation bias.” This may occur in the operators of autonomous platforms, depending 
on what level of autonomy exists. The stress of the war, though geographically distant, 
could still affect the human in the loop. This could also result in moral buffering because 
the human is so far removed from the actual “killing” that he might not consider the 
ethical consequences of his actions. While human warfighters may experience increased 
levels of psychological distance from the battlefield, completely autonomous platforms 


will not have any psychological understanding. As Krishnan recognizes, “machines will 
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have no idea what it means to be a human or a living creature and to suffer. They could 


inflict the worst suffering to humans without being emotionally affected in any way.”!44 


The emotional and moral disengagement could have strategic implications and 
affect America’s willingness to wage war. Krishan argues, “in robotic warfare the face of 
the other simply disappears and most likely also our ability to feel morally responsible for 
our enemies.” !45 The increased reliance on autonomous UCAVs could lead the nation to 
deploy force that otherwise might be attempted with greater and possibly more necessary 


reserve. 
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Vv. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 


A. SUMMARY 


Arguing that any use of violence is immoral does not solve the ethical 
questions and dilemmas of societies and professionals engaged in war and 
warfare. Wars are certainly bad in ethical terms, but wars fought without 
any ethical restraint are many times worse. The military and society 
cannot escape the question of how to fight and how to kill ethically, even if 
this sounds, maybe rightly so, quite distasteful to some people. 


Armin Krishnan, Killer Robots!46 


Clausewitz reminds us that, “theory will have to remain realistic. It cannot allow 
itself to get lost in futile speculation, hairsplitting, and flights of fancy.”!4” The question 
of autonomy and lethal force extends well beyond the question of military airpower and 
strike decisions, but also to the very identity of the warfighter and the nation’s character. 
Attention to ethics is not an impediment to embracing a vital strategic conclusion 
concerning the future of warfare, but rather the linchpin to moving forward in the right 
direction. Peter Singer argues, “we do have a choice about how we think about war, about 
killing, and about the value of human life in our society.’”’!48 While the issue of ethics and 
robotics may sometimes seem like science fiction, that perception should not be allowed 
to inhibit the necessary ethical reflection on the introduction of autonomous lethal force 
in unmanned platforms.!49 Our military must consider the future implications of 
unleashing this game-changing weapons technology. It is a debatable, considered choice, 
and not merely a foregone conclusion that the U.S. or its adversaries will develop 
autonomous UCAVs. The following recommendations are offered for the Navy and DoD 


as a practical approach to the future of military unmanned platforms. 
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B. CONCLUSIONS AND RECOMMENDATIONS 


Based on the preceding analysis, the following conclusions and recommendations 


are offered in response to the original research questions. 


1. 


To What Extent Do Contemporary Understandings of Military Ethics 
and the LOAC Address Machine Autonomy in Conjunction with the 
Use of Lethal Force? 


a. Conclusion 


“At present, there are no laws or treaties specifically pertaining to 


restrictions or governance of military robots [and] unmanned platforms.”!5° Joint 


Doctrine does address the deployment of unmanned aerial systems in a manner parallel to 


the Joint Doctrine on manned aircraft with only minor recognition of differences in the 


capabilities of remotely piloted aircraft. The LOAC does not currently address the 


autonomous use of force that may be used in UCAVs. Hartle identifies the 


incompleteness of the laws of war with respect to new military technologies: 


The incompleteness of the laws of war that results from the changing 
means of conducting warfare is particularly evident with respect to air 
warfare and aerial bombardment. Technological advances have been so 
rapid and so dramatic that the slow process of achieving consensus 
through usage has not kept pace...The existing laws of war appear to be 
regrettably incomplete in terms of providing guidance for the use of 


modem weapons. !5! 


The current laws of war are largely insufficient regarding the combination 


of machine autonomy in conjunction with the use of lethal force. Professional military 


ethics also do not specifically address the marriage of autonomy and lethal force. An 
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applied understanding of the just war theory can, however, offer a starting point to 
consider the ramifications of introducing lethal autonomous platforms into the force 


structure. 
b. Recommendations 


There should continue to be research and debate throughout the military 
and with civilian audiences on the challenges that emerging military technologies present 
to the Armed Forces. DoD policy analysts, and those who are responsible for writing 
updated doctrine, need to consider the ramifications of unmanned platforms on the 
battlefield and their implications for the introduction of autonomous platforms. There 
should be also case studies in professional military education that explore new challenges 
inherent with the use of autonomous UCAVs. The new psychological challenges to the 
Armed Forces should also be considered. The importance of ethical and policy 
discussions in the field of emerging technologies in professional military education was 
highlighted in recommendations on these matters contained in the Executive Summary 
for the 2010 McCain Conference. Specifically, regarding professional military education, 
they recommended: 

Collaborate with colleagues in engineering and technology to develop 
appropriate ethical modules for use in currently offered technical courses 
focused on military robotics, nonlethal weapons, warrior enhancement, 


and cyber warfare within undergraduate and graduate engineering and 
computer science departments 


Incorporate discussion of the latest emerging technological advances 
within the scope of current required core courses in ethics and military 
leadership, replete with case studies and concrete scenarios that 
distinguish competently between present fact, likely future prospects, and 
science fiction!>2 


Another important recommendation for moving forward in the field of 
autonomous platforms is to remain mindful of ethical consideration in the acquisition and 


design of these platforms. The unique opportunity to shape the future tactics and strategy 
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of wars exists not only for the individuals who are directing their research in the 
innovative field of unmanned platforms, but also and especially the force development 
personnel in the DoD. Their potential to influence future wars is a great responsibility 
and they need to consider more than costs as their decisions carry strategic implications 
on future warfare. The implications of the introduction of autonomous UCAVs or any 
autonomous platforms that deploy lethal force are on a scale and magnitude that, over 
time, can re-shape the force structure of the Armed Forces and affect political and 


cultural implications in America’s strategic decisions to wage war. 


A dialogue needs to continue and strengthen between the policy world and 
the world of engineering.!53 Everyone involved in the acquisition, development and use 
of UCAVs must recognize and consider the implications and far-reaching consequences 
of allowing autonomous platforms to be armed with lethal force. Dialogue between 
policymakers, the military, and engineers should positively impact the ethical decisions 
on the appropriate use of unmanned platforms. This shared responsibility will prove 
beneficial because, as Armin Krishnan notes, “one of the main factors that substantially 
contributes to moral disengagement is the diffusion of responsibility.”!5+ The continued 


dialogue and shared responsibility will avert any potential moral disengagement. 


Dee What Permissible Uses of Autonomous Ucavs Might Be Envisioned 
Within the Present Framework of Warfighting? 


a. Conclusion 


The nation’s Armed Forces are currently engaged, for the most part, in 
irregular warfare, which presents a very challenging ethical environment for the 
introduction of autonomous UCAVs. We may find ourselves, however, once again 
engaged in conventional campaigns. Conceivably, in such campaigns, “ground-combat 


power [may] be irrelevant; victory would come down to which nation could field more 
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advanced jets, ships, and other high-tech weaponry.”!55 A conventional campaign, 
possibly over open waters (as described above), may provide a ripe field for the 


introduction of autonomous UCAVs. 


There are different levels of autonomy that may be possible in UCAVs, 
ranging from a lower level (in which a pilot controls the aircraft remotely) to higher 
levels (where artificial intelligence increasingly comes to exercise sole control of the 
aircraft).!5° Drennan states that, “in this case, the tactician is neither the operator nor the 
robot, but the programmer. Regardless of the level of automation, however, robotic 
warfare will rely on the tacticians, whoever they may be, just as much as does 


conventional warfare.’’!57 


The military has long used technologies in new ways that were not 
originally envisioned in their development. The same thing may occur with the 
development of autonomous platforms and quite possibly on an even greater scale, owing 
to their artificial intelligence. Singer argues that this is already occurring today in 
unmanned systems: 

Robots in Iraq and Afghanistan today are sketching out the contours of 
what bodes to be a historic revolution in warfare. The wars of the future 


will feature robots of a wide variety of sizes, designs, capabilities, 
autonomy, and intelligence. 158 


The military may eagerly anticipate the possibility of autonomous UCAVs 
but should proceed with thoughtful caution and care. As outlined below, the legally and 
morally permissible uses of autonomous UCAVs within the current framework of war- 
fighting would likely include defensive postures, and semi-autonomous scripted offensive 


uses that retain some level of human control. 
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b. Recommendations 


(1) Where to automate? Several possibilities exist in partial 
automation of UCAVs, in which the operator’s judgment regarding appropriateness of 
lethality is not required.!59 For example, automatic-take offs and landings, flight 
checklists, and responding to certain flight emergencies do not demand the same ethical 
considerations as the decision to use lethal force in UCAVs. Efforts should be made 
toward automation when it increases the overall system effectiveness of the UCAV. This 
could enhance the operator’s ability to focus on more critical tasks such as the decision to 
deploy lethal force.!69 The following policies were recommended in the Executive 
Summary for the 2010 McCain Conference: 

Unarmed intelligence should precede strike missions, and initial strike 


missions should be restricted to nonlethal weapons and combatants-only 
areas, and include permission-seeking and override features. 


We strongly advise against incorporating “strong artificial intelligence” in 
such systems, which would render them capable of learning and even 
choosing ends, inasmuch as strong AI is highly likely to introduce 
unpredictability and/or mitigate human responsibility.1°! 


(2) Enhanced Virtual Environment. Proponents of the increased 





use of unmanned platforms argue that it is essential that the virtual cockpits are 
constructed to present the pilot with a much more real experience of flying in order to 
enhance their real time operating and strike abilities. This recommendation is applicable 
to the discussion on autonomy in UCAVs. Video games today are far more life-like than 
the systems used to control RPAs. Retired Air Force pilot and Predator Squadron 
Commander, Michael Keaton, argues for the need to create a compelling operator 


experience that keeps pilots and operators engaged and focused on the mission.!© This is 
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known as enhanced visual environment (EVE). The proposition may be less feasible with 
the introduction of autonomy in unmanned systems, but as long as there is some level of 
executive human oversight, there is the opportunity to design the control centers as 
environments that facilitate “immersive situational awareness that reduces operator 


workload for awareness.” !63 


One countering note: the EVE recommendation does not resolve 
the conflicting tendencies of a new distance for the war-fighter (described earlier on 
pages 44-46). Even in an enhanced virtual environment, the operator of an unmanned 
system does not take on a personal risk that he does in the actual theatre of war. Despite 
facilitating an enhanced environment, EVE does not eliminate concerns regarding the 


moral disengagement of combatants from the destructive force they are controlling. 


(3) Autonomy in other platforms. The DoD may introduce 
autonomy in platforms other than lethal UCAVs. There are opportunities besides UCAVs 


where the introduction of autonomy may not raise the same ethical concerns. For 
example, the successful use of robotics to identify and destroy IEDs has already saved 
American lives. Similarly, there is incredible potential for autonomous platforms that 
detect and destroy sea mines, which continue to present a grave threat to the Navy. Mine 
warfare is proving to be one of the military’s most vital force components. The potential 
for autonomous platforms to serve the nation’s strategic interests in mine warfare 
provides opportunities without the same ethical concerns that affect urban warfare and 


the deployment of lethal force from a UCAV. 


(4) Strategic force structure. The introduction of organic force 
structures that deploy traditional piloted aircraft alongside unmanned systems with 
varying degrees of autonomy will affect force structure and manpower requirements 
throughout the nation’s Armed Forces. One of the main force structure considerations 
that may change with the introduction of autonomous UCAVs will be training pipelines. 
Manning requirements and timelines for flight school and airmen training will certainly 


be affected, but the changes will go well beyond those domains. Ratings and designators 


163 Keaton, “UAS Operators and Autonomy.” 
49 


will need to adapt to integrate into the new organic force structure. Consequently, 


training and manning requirements will affect recruiting and raise questions such as: 


e What kind of individuals will the military seek to fulfill these new jobs? 


e Can the level of manpower remain the same or will it decrease or increase 
and by how much? 


e How will the changes in manpower requirements, due to the increased use 
of unmanned systems, and the introduction of autonomy affect costs? 


The Armed Forces may discover that the increased use of 
unmanned platforms and particularly automation eliminates many of its most challenging 
manpower problems, simply by eliminating the manpower. Krishnan argues, “reliance on 
automated systems could mean that defense could be organized more cheaply and in a 
purely defensive posture.”!©4 Conversely, the Armed Forces may discover the opposite. 
Costs may be higher with the increased use of autonomous platforms. While the force 
structure and manning requirements cannot precede the DoD policy decisions that 
determine which, if any, autonomous platforms are built and implemented, we must 
anticipate these very real possibilities and evaluate possible force structure changes. We 
need to optimize the ideal human footprint that will co-exist alongside these new 
platforms. Cost decisions are important and will drive these decisions to an extent. Cost 
considerations alone, however, should not be allowed to dominate force structure 
considerations. We must remember that the professional decorum of the Armed Forces 
and the success of our future operations are dependent on upholding our time-honored, 
professional commitments to the ideals of the Just War Tradition, as well as force 


structure efficiency. 
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3. Is It Necessary to Retain a Human Presence, or Executive Oversight, 
in the UCAV “Kill Chain?” 


a. Conclusion 


It is necessary to retain, at minimum, executive oversight in the UCAV 
“kill chain,” particularly in combat aerial support in irregular warfare. UCAV missions 
today involve multiple eyes on target before strike decisions are made. These real-time 
decisions require an operational awareness from both the ground and the air that regularly 
includes last minute information from ground forces that influence a commander’s 
decision to call in air strikes from a UCAV. A human warfighter will still be needed “in 
the loop” to determine if “killing” is permissible under an applied understanding of the 


Just War Tradition and the Laws of Armed Conflict. 
b. Recommendations 


(1) Failsafe control. There should be, as Peter Singer aptly 
describes it, a “failsafe human control option” in an UCAV that has autonomous 
capabilities.!©5 Rather than eliminating the human decision-maker in the “kill chain,” the 
Executive Summary of the 2010 McCain Conference included a recommendation to 
improve their decision-making ability: 

Increased reliance on unmanned systems—both remote-controlled and 
autonomous—will require better informed and monitored decision-makers 


in order to ensure compliance with traditional ‘just war’ requirements, 
such as “last resort” and “legitimate authority.” 166 


For example, in the design of the UCAV, an operator should be 
given enough options to stop a lethal strike. The human operator, who remains in the loop 


even with minimal executive oversight, must have the ability to make a last-minute 


165 Singer, Wired for War, 424. 
166 Barrett, “Executive Summary,” 424-431. 


51 


decision to avoid a wrong turn, for example, that results in firing on a children’s soccer 


field that was improperly identified as a target by the autonomous system. !©7 


(2) “Concluding Unscientific Postscript.” The greatest challenge 





to the proposition that autonomous platforms will make better “kill” decisions than 
human pilots or operators is not a problem with unmanned systems or even the increased 
reliance on artificial intelligence. As discussed throughout this research, there are many 
advantages to be gained by the Armed Forces’ use of unmanned platforms and 
autonomous technology. The difference between a human pilot flying an F-18 and firing 
a missile, and a remote operator at Creech AFB flying a Reaper and firing a weapon may 
be miniscule from an ethical standpoint, provided the operators have the proper training 
and situational awareness to ensure they are making ethical strike decisions. The direct 
implications of completely autonomous UCAYs that cannot be avoided or “managed” 
occurs when we turn from targeting the enemy’s weapon systems to deliberate targeting 


of other persons. 


Before autonomous platforms that make their own decisions to 
kill human beings are introduced in battle, we need to prudently consider the grave 
ethical ramifications of unleashing this game-changing military technology. Unmanned 
and semi-autonomous platforms may bring significant advantages to the Armed Forces 
and enhance our abilities for mission accomplishment, but a boundary needs to be drawn 
before “robots” are allowed to decide whether to kill other human beings. Kant’s 
“celebrated practical categorical imperative—Act so as to treat humanity, whether in your 
own person or in that of another, always as an end and never as a means only” should not 


be forgotten. 168 


There may be an intangible and fundamental element of a human 
being that is vital and irreplaceable to the military profession. Until we resolve that 


question, we may not readily duplicate the “kill” decision in a machine. A philosophical 
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current called personalism, “an approach or system of thought which regards or tends to 
regard the person as the ultimate... principle of all reality,” might guide the Armed Forces 
in this question.!©? Karol Wojtyla demonstrates how moral actions are “where we find 
the center of the human person, the core of our humanity.”!7° This philosophical current 
is applicable to the present question because our discussion on what military actions may 
be permitted by an autonomous UCAV is predicated on what is ethical or morally 
permissible in keeping with the primary mission of the military profession. Our 
continued efforts to act in an ethical manner are inherent in the profession of arms, as 
well as in our commitments toward mission accomplishment and securing peace. !7! 
Moral actions are inextricably intertwined with our humanity and we cannot transfer this 


responsibility and our professional moral identity to artifacts and mere machines. 
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